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Digital information transmitted by means of a pulse amplitude modula- 
tion scheme depends critically on the pulse shape for reliable high speed 
communications. The pulse shape, in turn, depends in great measure on 
precise phase equalization. A new technique for the design of phase equal- 
izers, based on digital mean square error, has been developed. This criterion 
is appropriate for a digital transmission system because it can be related 
to the system error rate. 

A lower bound to the digital mean square error is first obtained by 
determining the theoretically optimum phase. A physical equalizer consists 
of a cascade of many (say N) constant resistance all-pass networks. For 
each of several different values of N, an optimization search over the 
parameters of the all-pass networks is then done. The smallest value of N 
which yields an error satisfactorily close to the lower bound is utilized for the 
optimum physical phase equalizer. 

The major benefits derived from using this technique as opposed to the 
conventional one are: 

(i) A significant reduction in the number of all pass sections required, 
(ii) More practical element values, facilitating network manufacture. 

(Hi) A substantial improvement in system performance. 

I. INTRODUCTION' 

Most information transmitted via telephone facilities today is sent 
over analog channels. It is anticipated, however, that future systems 
will be largely digital. New services such as Picture phone & will be 
transmitted over toll facilities in a digital format. The Bell System will 
need to provide a significant digital capacity by the late 1970s. In spite 
of this emphasis on digital transmission, analog demands will be rapidly 
increasing for some time to come. The L-4 system, originally conceived 
to provide increased analog capacity, is now being equipped to transmit 
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in a digital mode as well. L-4 can therefore temporarily satisfy increasing 
analog and digital requirements. The eventual transition from analog 
to digital can be accomplished smoothly by apportioning analog and 
digital usage according to demand. Similarly, the L-5 system will be 
able to handle both analog and digital signals. Digital transmission 
on L-5 is expected to be available earlier than any purely digital system 
of comparable capacity. These systems with their dual capabilities 
promise to play an important role in the future of Bell System long-haul 
transmission. 

The scheme (Fig. 1) employed for transmitting digital information 
over L-4 and L-5 is pulse amplitude modulation. The sequence of input 
digits, {a*}, is used to scale translates (displaced by T seconds) of a 
given pulse shape. If these pulses do not overlap, then their amplitudes 
can be determined from samples of the received waveform (assuming 
noise free transmission) and the scale factors ascertained. Efficient use 
of channel capacity dictates simultaneous transmission of several 
signals, each confined to a specific bandwidth. The fulfillment of these 
bandwidth restrictions causes the transmitted pulses to overlap, and 
complicates the process of extracting the information digits. For band- 
widths of greater than 1/2 T (for single sideband), there exist a class of 
pulses, known as Nyquist pulses, which allow, at one point in each T 
second interval, for the extraction of an interference-free sample. 
Unfortunately, it is not possible to build filters to realize these pulse 
shapes exactly or to build sampling equipment with zero pulse widths. 
Consequently, one is left with an unavoidable amount of "intersymbol 
interference" such that the output sequence, {b k \, will not be an exact 
duplicate of the input sequence, {a k }. 

The niters have a second function no less important than pulse 
shaping, namely, to prevent serious interference between signals in 
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Fig. 1 — Model for the digital communication system. [V(f) is the transfer charac- 
teristic from 1 to 2.] 
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different bands. The filters must satisfy out-of-band rejection require- 
ments as well as yield desirable pulse shapes. 

With traditional filter design techniques, phase distortion is not 
considered initially.* An amplitude characteristic, fulfilling out-of-band 
rejection requirements, is realized as closely to a Nyquist shape as 
possible. Then the phase is modified by means of a cascade of all-pass 
sections so as to minimize the "curve fit" mean squared difference 
(over the pertinent bandwidth) between the resulting delay and a 
constant delay. This method (of reducing curve fit mean squared error) 
was initially used in L-4 for the design of filters for digital transmission. 
The result proved to be unacceptable. The major difficulty with this 
procedure is the fact that the resultant phase distortion, as determined 
above, cannot be readily related to an error between an input symbol, 
a k , and the corresponding output, b k . That is, not all errors of equal 
size as measured by this criterion affect the digital error to the same 
extent. Any criterion which does relate to differences between the 
{a k ) and the {b k } would necessarily consider both amplitude and phase 
simultaneously. The digital mean square error, to be defined in the next 
section, is a criterion which is appropriate for our purposes and is 
related to the differences between digital input and output. 

Although it was originally planned that the amplitude and phase 
equalization would be designed simultaneously, it was found to be 
advantageous not to do this initially. The problem becomes much more 
manageable by separating amplitude and phase designs into two inter- 
acting parts. In some cases, it was possible to achieve satisfactory 
amplitude characteristics by traditional methods. For these cases, the 
problem becomes one of designing a realizable phase characteristic 
approximating the optimum phase (which depends on the amplitude 
and which is not necessarily linear). Therefore, the first new effort is 
the development of a new technique for phase equalization using a 
criterion relevant to digital transmission. 

II. DIGITAL MEAN SQUARE ERROR 

The digital mean square error, D k , of the A;th digit is defined as the 
mean squared difference between the input, a k , and its corresponding 
output, b k 2 " 3 ; that is, 

D k = E{(a k - b k ) 2 } (1) 



* In his paper, "Synthesis of Pulse-Shaping Networks in the Time Domain," 
D. A. Spaulding does consider amplitude and phase simultaneously. 1 
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where E{ • } denotes expectation with respect to the random variables 
a k and b k . If the input sequence is wide-sense stationary and the channel 
is deterministic, conditions which we shall properly assume, then D k is 
independent of k, and the subscript k will be dropped. 

It can be shown (see, for example, Ref. 4) that equation (1) can be 
written as 

D - Too + f •!/(/) V*(fi exp G'2ir/r) 

J-oo 

■ftjiK'-f)-*[-*('-f)']- 8 }« (2) 

where 

V(f) is the transfer characteristic between points 1 and 2 of Fig. 1, 
r is a constant representing a delay in the sampling time, 
T is the pulse repetition time interval, 
and 

M(j) is a factor arising from the correlation between input digits { a k \ . 

If we define 

Pn = E{a k a k+n \ = p-„, (3) 

then, assuming convergence, we have for M(j), 

M(D = E Pn exp (-j2irfnT) =Effl, cos 2TrjnT (4) 

where we have denned 

ra = p 
and 

m n = 2p„ , » 9* 0. (5) 

The error, D, can be written in a more convenient form if we define 
an error term, «(/), as: 

•CD = \ £, F(/ - f) exp [-i2x(/ - f)r] - 1. (6) 

Using equation (6) in equation (2) and noting the fact that the imaginary 
part of e(/) is odd, we can write D as: 

D = m + T f /T M(j)[*(f) ~ !)[<*(/) + 1] rf / 

J-1/2T n\ 

= 2T f /2T M(f) | e(j) | 2 dj. 
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The function e(/) is a measure of intersymbol interference. If the 
pulse shape were Nyquist, «(/) would be zero. The term M (/), due to 
the correlation of the input digits, acts as a weighting function in the 
error expression. 

c(/) is determined from the transfer functions of the block diagrams 
in Fig. 1. We can take advantage of the linearity of the vestigial sideband 
system and, by translating appropriately, lump several filters together. 
We define, therefore, the frequency transfer function for the combination 
of the two low-pass filters as L(/), for the combination of two bandpass 
filters as B(f), and for the combination of the all-pass networks (that is, 
the phase equalizer) as A(j). The cable or analog system is represented 
by C(f). With this notation, we can write the baseband equivalent to 
the passband portion of the system as: 

H(j) = hi A (i + f c )B(j + QC(j + //) exp (jd) 

+ A(! - f e )B(j - QC(j - j c ) exp (-jd)} rect (f ) (8) 

where 

rect (.«•) = j 1, 1*1 <* (9) 

lO, otherwise 

and where /„ is the carrier frequency and 6 the phase offset (the difference 
in phase between the modulation and demodulation cosine functions). 
The overall transfer function becomes simply 

V(j) = L(i)H(i). (10) 

In general L(f), /?(/), and C(J) are not measured separately. Instead 
R(f) is measured where 



/«"(/) = 



±L(j-1 r )B(fiC(1), /> 
JjL(j + f e )B(j)C(j), f <0 



(ID 



with the result that 

V(f) = T[R(j + L)A(j + j c ) exp(^) 

+ R(j - U)A(j - /,) exp (-jd)} rect (^)- (12) 
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If we define F,(/) as 

Pifl)-£ 7(flexp(-j2r/r), (13) 

where g is a scale factor, then e(j) becomes, in terms of Vi(j), 

<f) = t i V l (f - |) - 1 (14) 

where the limits on the sum reflect the bandlimited nature of V(J). 

Since we are initially concerning ourselves with phase equalization 
only, we want to consider R(j) as fixed and modify c(/) by our choice 
of A (J) where | A(J) | = 1. The optimum design for A(j) is considered 
next. 

III. OPTIMUM PHASE 

In order to obtain a lower performance bound to the optimum design 
of the phase equilizer, it is necessary to determine, for a given amplitude 
shape, the phase associated with the minimum error, D. For this 
purpose, once again consider the error term, 



V 1 (f - |) cxp (jfc(/ - f)) ~ 1 (15) 

where <h(/) is the phase of F,(/). 



e(/)= E 



Because V^f) is zero, for | / | > ™ , then for ^ / ^ ^ 



we have 

i <f) r 



= | I V t (f) I exp (jhfl)) + I V t (j - |) exp (j</>,(/ - |)) - 1 
- [| 7,0) | cos <*>,(/) + | 7,(/ - |) j cos*^/ - ^) - I]' 

+ [| V.tf) I sin 0,0) + | F,(/ - |) J sin <*>,(/ - £)J ■ (16) 

Clearly, minimizing | e(/) | 2 for each / minimizes D. Let us use the 
following shorthand notation 
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and 



y = cos^/ - -j , 
V = I Vx® I. 



7 r = 



»M/-j 



Then for / fixed we wish to minimize 

J(x, y) = (Vx + V T y - l) 2 + [7(1 - x 2 )* + 7,(1 - y 2 )*} 2 (17) 
subject to the constraints 

| x | g 1 and | » | ^ 1. (18) 

Let F be the feasible region, that is, the set of points (x, y) which satisfy 
inequality (18). Let F be the interior of F. Since J(x, y) is differentiable 
in F, then the optimum point (£, y) is in F if and only if 



3J(x, y) 
dx 



(i.u) 



dJ{x, y) 
By 



= 0. 



(I. V) 



(19) 



If (£, #) lies on the boundary of F then equation (18) is not necessarily 
satisfied. 

We can now state the major result concisely as a theorem. 



Theorem: If 

V + V T > 1 and \V - V T \ < 1 
i/ien (£, $) e F and a) a??rf # are determined by 

1 + (7 + F r )(F - 7 r ) 



.f = 



and 



yielding an error 



2V 



// = 



_ 1 + (7, + F)(7 r - V) 



(20) 



(21) 



27. 



./(*, V) = 0: 
however, if (20) is ??o£ satisfied then 

£ = 1, 0- -1, if 7^ 7, 



(22) 
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or 

x = 1, t? = 1, if V < V T . (23) 

Proof: If (x, y) tF then equation (19) holds and solving for x and y we 
obtain equation (21). We must show that if inequality (20) holds, 
then equation (21) is consistent with (x, y) t F. In order to do this, 
we define a and A as: 

, = V + V T - 1 (24) 

and 

A = 1 - (V - V T ). (25) 

Now inequality (20) implies that a > and < A < 2. Therefore for 
a > and < A < 2 we must obtain the values of x and y such that 
| x | < 1 and | ?/ | < 1. Using equations (24) and (25) in equation (21), 
we obtain for £ 

= 1 + (1 + q)(l - A) 

r 2 + a - A 



= i - ' '• 



2 + a - A 



Since we have 



< - _,/* . < 2, (26) 

2 + o- — A 



then |i| must be less than one. Similarly, we obtain 



= 1 - 



1 - (1 + «r)(l - A) 

a + A 

2 - A 



* = a + A 



(27) 



1 + (A/a) 

Since the quantity 1 + A/cr is greater than 1 and < 2 - A < 2, then 
| y | < 1, proving that if inequality (20) is satisfied, then (x, y) c F. 
Substituting equation (21) into equation (17), we get 

./(-v, fl) 
- » + KV + V,)(V - V r ) + ft - |(7 + V r )(V - V T ) ~ I] 2 

ay* - i - 2K 2 + 2n- (v 2 - nry 



+ 



4F 2 r - 1 + 2V 2 - 2V 2 T - (V 2 - V\) 



2 

r 2 \ 2 \h 



= 0. (28) 
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If inequality (20) is not satisfied, then the point (x , y) is on the boundary. 
In order to minimize J(x, y) we must eliminate the second term in J; 
that is, we must have | x \ = \ y \ = 1. If V ^ V T then x = 1, y = — 1 
minimizes J; otherwise x = —1, y = 1 minimizes J. 

The optimum phases <£,(/) and </>i(/ — 1/T) are obtained from the 
equations 



<£,(/) = cos 1 x, 



and 



*\ f --f)'\ _cos " '" y>0; (29) 

I— k + cos ' y, y < 0. 

In this manner, the optimum phase is determined for all / for the base- 
band signal Vi(j). Except in the Nyquist roll-off region, 0(/) is zero. In 
the demodulation step, there is a similar overlapping in the vestigial 
roll-off region and an identical procedure can be done provided the 
two roll-off regions do not overlap. If they do, then the process is nearly 
the same except that one must first determine, for the overlapping part 
of the vestigial region, a phase difference between the terms that add 
together after demodulation and then determine the phase of the base- 
band waveform. The solutions are, in general, not unique. 

IV. REALIZABLE PHASE EQUALIZER 

The optimum phase, as specified in the previous section, can not be 
realized. For physical systems we are restricted to those phase charac- 
teristics that can be achieved by cascading sections of constant resistance 
all-pass networks. A typical constant resistance all-pass network is shown 
in Fig. 2. Its transfer characteristic is 

E a /E t = exp (-#(/, /„ , &„)) (30) 

where (Fig. 3) 

P(j,1 n ,b n ) = 2tan-'|-^-pj- (31) 

Using N such all-pass networks yields an overall response A (/) , such that 

A(j) = exp (- j g /3(/, f n , &„)). (32) 

Substituting equation (32) into equation (12) and using measured 
characteristics for R{j), we obtain F,(/), e(j) and, in turn, D in terms 
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Fig. 2— Two realizations of a constant resistance all-pass network. 
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of the b n and /„ . The next step is the optimization search to choose the 
set of 6„'s and / n 's to minimize D. 



V. THE MINIMIZATION OF D 



Since D is now seen to be a function of many parameters, it is more 
appropriately written as 

D(f, b, r, g, 6) - 2T f /T M(j) | e(j, f , b, r, g, 0) \ 2 df (33) 

•'0 

where Vi(j) in equation (14) is equal to gV 2 (j) and 



V,(j) =\\R(J + /.) | exp 



;</>(/ + U) ~ E M + 1c,U,b n ) + d 



+ R(-1 + 1 C ) exp 



-j[4>(1 + /.) - Z «-f + /-./-. &-) + 



• exp (j'2it/t) rect (^yj : 



-)]] 



(34) 



<7 is a scale factor for F, (/) ; b and f are parameter vectors 



b = 



and </>(/) is the phase of R(j). 

If we adopt the following inner product notation 

[«(/), /3(/)]„ = 2T f /T M(j)a(j){J*(j) df, 



M 






b 2 


, f ■ 


u 


X. 




Jn. 



(35) 



(36) 



then D can be concisely written as 

D(t, b, r, fir, fl) = [e(/, f, b, r, (7, »), e(/, f, b, r, g, 6)} M . (37) 

We must first minimize D with respect to the gain, g, the delay, t, 
and phase offset, 9, before determining what changes to make in b and f . 
After changing b and f, we repeat the procedure by optimizing g, t, 
and 8, and then obtaining new b and f . After each iteration, we evaluate 
D and stop when D ceases to improve more than a predetermined 
amount. 

To ascertain the optimum value for g, we solve dD/dg = for g 
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and obtain 



Re 



E vAf 



-*).']. 



it '■( 



f )■£. 



r, 



/-f 



(38) 



g does depend on the phase of V 2 (f); however, if the roll-off region is 
narrow, g is relatively insensitive to it, especially when b and f are near 
their optimum values. We can, therefore, eliminate repeating the cal- 
culation of g at each iteration by replacing V 2 (f) by \V 2 (j) | in equation 
(38) to obtain 



T r , 



'-!)!• 4 



™ - f • .?, 



r s 



f TV _,, 



(39) 



which, since it is independent of phase, need be calculated only once. 
We shall take a similar approach for the evaluation of 6. Optimization 
with respect to 6 would have to be done jointly with optimization with 
respect to r. This would enormously complicate the minimization of D, 
since it would have to be repeated at each iteration. Instead we can 
choose as 



6 = -0(/«) + E 0(f. - /. . to + 0o 



(40) 



where O can be chosen so that 8 is very close to its optimum value for 
the final set of b and f. Thus, as b and f improve, the value chosen for 
approaches its optimum. 6 depends on R(f) and is determined from the 
optimum phase. 

With g and 6 fixed, the optimization with respect to t has been 
simplified. T opt is a solution of the equation 



OT 



= 2 Re 



e(/,f,b,r), 



de(f, f,b, r) 
dr 



e(/,f,b,r), ZMl-§) V \f-§ 



= 0. 



(41) 



An initial value of t is chosen as described below and dD/dr is evaluated. 
If dD/dr > 0, then r is decremented, or if dD/dr < 0, then r is incre- 
mented, until dD/dr changes sign. Using the last two values of dD/dr, 
we can employ the method of "false position" (regula falsi) 5 to deter- 
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mine a value of dD/dr < 10 -5 . The t yielding this value of dD/dr is 
accepted as r opt . 

t, , the initial value of r, is chosen to correspond to the best linear 
mean square error curve fit to the resulting baseband phase. Thus we 
have 



where 



•• = -m/2t 



Z (ft ~ 1cH(U) 
" " ^ (42) 

S (A - /,) 2 



A 1 



H1) = <*»(/) - Z /S(/, /« . &») + e, 

and the f k are the discrete values of frequency between / = and / = 
1/2T used in evaluating D by a finite sum. 

The minimization is done by a parameter search utilizing the Fletcher- 
Powell algorithm. 0,7 Since the effectiveness of the method depends on 
the accuracy with which the gradient is determined, it is particularly 
advantageous to use an analytic expression for its calculation. This 
expression is presented in Appendix A. 

The computer program to achieve the above minimization was written 
in its entirety by Mrs. Barbara E. Forman for the CDC 3300 and the 
IBM 360/75. 

Practical problems can arise in performing the parameter search. 
They are due mostly to the existance of many local minima. The 
Fletcher-Powell algorithm will approach a local minimum as readily as 
a global minimum. This difficulty is mitigated by three factors: (i) We 
have been able to determine, for a given amplitude characteristic, the 
theoretical phase which will yield the lowest value of D and thereby 
determine the practical utility of trying to find the global optimum — we 
have been able to find local minima which yield digital mean square 
errors insignificantly greater than the lower bound, (ii) We are able to 
specify the largest acceptable value of D. (iii) The parameter values at 
the global optimum may be impractical to realize by physical networks. 

The system error rate objective for L-4 and L-5 is that the probability 
of error (P F ) be less than 10" 8 per regenerative section. Walter E. Norris 8 
has been able to relate a P E of 10~ 8 to a value of D of approximately 10" 3 . 
This relationship is not unique, but it is accurate to within a practically 
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acceptable tolerance. We set our requirement for the theoretical filter 
at D = 5 X 10~ 4 since the physical realization will result in a larger 
error. We will accept a larger D for the implemented filter provided its 
value for D is reasonably close to this. If a minimization design yields 
a value of D near the lower bound and below the requirement, it is of 
no serious consequence that we do not achieve the global minimum. 
Since our design is based on a mathematical description of the network, 
it is important, however, that the parameter values be such that the 
physical realization approximates closely its mathematical representa- 
tion. It is possible for a local optimum in a favorable parameter region 
to produce a better result than the global optimum in an unfavorable 
region due to attendant amplitude distortion and problems in its 
manufacture. 



VI. COMPARISON OF DESIGNS 

The data available from the L-4 system affords an opportunity to 
compare the conventional techniques of phase equalization with our 
method. The digital sequences on the system will be encoded in a partial 
response 9 format which yields an M(j) of M(j) = 4 sin 2 4t/7\ The 
amplitude characteristic of the cascade of filters for digital transmission 
is shown in Fig. 4. By conventional methods of phase equalization, 
20 all-pass sections were designed to reduce the phase deviation (devia- 
tion from linear phase, see Appendix B) to the "best achievable" level 
across the entire bandwidth. The theoretical performance of the 20- 
section equalizer (which does not consider the amplitude distortion of 
the phase equalizer) yields a phase deviation as shown in Fig. 5 and a 
digital mean square error of 1.45 X 10~ 4 . The "b" parameters (a 
measure of the steepness of the slope of the phase characteristics of an 
all-pass section, see Fig. 3) of several all-pass sections had values well 
above 25, the highest being 34.39. Values of b above 25 are undesirable 
for purposes of manufacture. The actual phase deviation as measured 
by J. S. Ronne 10 is shown in Fig. 6 and yields an error of 1.05 X 10~ 2 . 

An optimum phase deviation for the system, as determined by the 
method described in an earlier section, is shown in Fig. 7 and yields a 
digital mean square error of 5.69 X 10~ 5 . If the phase had been exactly 
linear, the error would be 7.52 X 10~ 5 . Our program will attempt to 
achieve the optimum phase. 

We must first determine the number of sections necessary to yield an 
acceptable error. In Fig. 8, the error, D, for the optimum design is 
plotted as a function of the number of all-pass sections. The curve shows 
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Fig. 4 — Amplitude characteristics of cascade of filters. 

that one obtains diminishing returns after 10 sections and that these 
10 sections have a theoretical error (1.41 X 10~ 4 ) less than that of the 
original 20-section design. Taking into account the reduced amplitude 
distortion in realizing 10 sections as opposed to 20 sections, and a 
minimal theoretical improvement in going from 10 to 20, we would, in 
all likelihood, achieve better performance from the optimum 10-section 
design than from the optimum 20-section design. A comparison of 
calculated amplitude distortions is presented in Fig. 9. It indicates that 
it is very desirable to use as few sections as is possible. 

The optimum 10-section phase equalizer (all of whose 6 parameters 
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Fig. 5 — Theoretical phase deviation of original 20-section design. 
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Fig. G — Phase deviation of original 20-aection design as determined from mea- 
sured characteristics. 
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Fig. 7 — Phase deviation of optimum phase. 
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Fig. 8 — Minimum digital mean squared error obtained with various numbers of 
all-pass sections. 
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Fig. 9 — Calculated loss characteristics of non-ideal phase equalizers. 
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Fig. 10 — Theoretical phase deviation of 10-section optimum design. 
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Fig. 11 — Phase deviation of optimum 10-section design as determined from 
measured characteristics. 
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Table I — The Digital Mean Square Error For The Two Phase 

Equalizer Designs 





Theoretical 


Measured 


Original Design 
(20 Sections) 


1.45 X 10" 4 


1.05 X 10~ 2 


New Design 
(10 Sections) 


1.41 X 10- 4 


1 . 99 X 10" 3 



were equal to about 20) was built and tested. The phase deviation of 
the theoretical design is shown in Fig. 10 with a corresponding error 
of 1.41 X 10~ 4 , while the phase deviations determined from the 
measurements of J. S. Ronne are shown in Fig. 11. This corresponds to 
an error of 1.99 X 10 -3 . Table I summarizes the results. 

A P E test was made on both the 10-section and 20-section designs at 




20 SECTION 
.PHASE EQUALIZER 



10 SECTION 
- OPTIMUM 
PHASE EQUALIZER 



30 32 34 36 38 

S/N RATIO IN DECIBELS 



Fig. 12 — P K vs S/N for the 10- and 20-section phase equalizers. 
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a signal-to-noise (S/N) ratio of 40 dB. The error rate of the 20-section 
was 2000 times that of the 10-section. The two designs were also tested 
under the circumstances in which they would be used. Attached to the 
system is an adaptive equalizer which automatically adjusts to partly 
compensate for distortions in the system. The P E of both designs were 
taken as a function of S/N, with the adaptive equalizer included, and the 
results are presented in Fig. 12. These measurements indicate that 
the new design yields 3 dB more S/N margin at P E = 10" 8 than the 
previous design. The lower bound on P E obtained from additive noise 
considerations (assuming a Nyquist pulse shape) by F. S. Hill 11 indicates 
that there is little room for additional improvement through manipula- 
tion of the pulse shape. 

The advantages of the new design can be summarized as follows: 
(i) it is less expensive (fewer sections), (tf) it is easier to manufacture, 
and, (in) most importantly, it yields better performance. 

VII. CONCLUSIONS 

The use of the digital mean square error, D, as a design criterion has 
proved to be of significant value in the design of the phase equalization 
for the transmission of digital information over the L-4 channel. Further 
studies indicate that comparable advantages exist in applying this 
technique to L-5 and other L-4 network designs. The use of the digital 
mean square error criterion is not limited to the phase equalization 
described here. Not mentioned above, but necessary, is amplitude 
equalization to compensate for the amplitude distortion introduced by 
the physical filters and all-pass sections. The design of this equalization 
is done optimally using the same techniques. 

With conventional techniques, the design of the band limiting filters 
considers amplitude characteristics alone. The passband of these filters 
should be designed to minimize D, in which case both the amplitude and 
phase influence the result. The next step, therefore, is the jointly opti- 
mum design of these filters, together with equalization, on a digital 
mean square error basis. 

VIII. ACKNOWLEDGMENTS 

The author is pleased to acknowledge the significant contributions of 
many people in this effort. In particular, the author wished to express 
his appreciation to Dr. R. E. Maurer for his encouragement and enthu- 
siastic support, to Mrs. B. E. Forman for her excellent work in writing 
the computer programs, to E. Miles for building and measuring the 



OPTIMUM PHASE EQUALIZATION 



1551 



equalizers, and to T. H. Simmonds, Jr. and F. R. Bies for many helpful 
discussions. 

APPENDIX A 

The Gradient 
The error D can be written 

D(t, b) - 2T f " '* M(j) | «(f, f , b) | 2 <*/ (43) 

where, for the band limited V(j)[ | V(j) | = 0, | / | > l/T 7 )] 

e (/) = F,(/) + V t {f - |) - 1 (44) 

and 

Vi(fl - (7{| R(1 + « | exp [;(«(/ + fj - g j8(f + f. , f. , K) + 5) J 

+ I R(-f + /c) I 
•exp [-;■(*(-/ + /.) - Z 0(-i + /.,/.. &») + 

•exp (j27r/T)rect ^— J- 
The gradient G is 

36, 

dD 
db 2 



G ■ 



3D 



(45) 



3/.V. 



1552 THE BELL SYSTEM TECHNICAL JOURNAL, SEPTEMBER 1970 

where 



3D 



ri It [••'-' -I 



dVfd) dl '•' T; 



2^ = 4T J M(/) Re jetf) 




(46) 



and 



37, 



37, 



d/« 



= Q exp 0'27r/r) 



I R(f + 1c) I exp (jjcK/ + /,) - *(/.) 



£ [/3(/ + /. , /, , 6.) - Mc , U , &n)] + *0J 



■; 



[/*(/ + /,)] 2 + f ft - Cf + /.) 2 ] a a,/,) 2 + f ft - ffl 5 



bji + / r )r/ 2 + (/ + /c) 2 i 



W,(tf + / 2 



[/»(/ + /J] 2 + f ft - (/ + /,)T (/,,/c) 2 + J f/ 2 - ffi' 

+ I R(~f + /c) I exp (-;{*(-/ + /„) - *(/,) 
- £ [£(-/ + U . /-. , K) - /3(/ c , /„ , &„)] + * 

n- I 

/„(-/ + f t )K-f + /,) 2 - 1l\ 



■j 



UXIl - ft 



[/,(-/ + /J] 2 + f Ul - (-1 + /J 2 ] 2 a/c) 2 + I ft - / 2 ] 



-&„(-/ +/ f )[/n+(-/+/ r ) 2 ] 



+ 



bJXf: + 11) 



[/„(-/ + u)f + f t/ 2 - (-/ + /,) 2 ] 2 ' (U) 2 + f ft - / 2 ] 



■rect 



2 



(47) 
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APPENDIX B 

Phase Deviation 

The phase deviation is denned as the deviation of the phase from the 
linear phase which best (in a mean square error sense) fits the phase. 

Therefore, if \f/{j) is the phase and af + b is the best linear fit between 
/i and / 2 then the phase deviation D is 



4> D (j) = m - (af + b) 



where 



a = 



'" U(S) df - r - 1 T 



r ' mdf)(f f ' fdj 



'><*/- 1 



fdf 



and 



6 = 



] 



/, - /. V,, 

Mdf-af'fdfj- 



U - U Jf, 
Discretizing the above equations, we obtain 

/A-,+n Wfc 1+ » 

E um - 



a = 



n 



Zfi- 



k,+n \2 

E 

i-i, 



n 



E *(/*) - a E /* 

i _ t-frl A— frl 

where/*, = /i, /*,+„ = / 2 and 

A/ = /*, + „ - /*,+„-, = 

REFERENCES 



n 



1. Spaulding, D. A., "Synthesis of Pulse-Shaping Networks in the Time Domain," 

B.S.T.J., 48, No. 7 (September 1969), pp. 2425-2444. 

2. Tufts, D. W., "Nyquist'a Problem — The Joint Optimization of Transmitter and 

Receiver in Pulse Amplitude Modulation," Proc. IEEE, 53, No. 3 (March 
1965), pp. 24S-259. 



1554 THE BELL SYSTEM TECHNICAL JOURNAL, SEPTEMBER 1970 

3. Maurer, R. E., "The Optimal Equalization of Random Channels," Ph.D. thesis, 

Northeastern University, 1968, pp. 231-238. 

4. Berger, T., and Tufts, D. W., "Optimum Pulse Amplitude Modulation, Parti, 

IEEE Trans, on Information Theory, IT-IS, No. 2 (April 1967), pp. 196-208. 

5. Hildebrand, F. B., Introduction to Numerical Analysis, New York: McGraw- 

Hill, 1965, pp. 446. 

6. Fletcher, R., and Powell, M. J. D., "A Rapidly Convergent Descent Method of 

Minimization," Computor Journal, 6, No. 4 (July 1963), pp. 163-168. 

7. Kowalik, J., and Osborne, M. R., Methods of Unconstrained Optimization Prob- 

lems, New York; Elsevier, 1968, Sec. 3.7, pp. 45-48. 

8. Norris, W. E., Unpublished work. 

9. Franks, L. E., Signal Theory, Englewood Cliffs, N.J.: Prentice-Hall, 1969 

pp. 219-220. 

10. Ronne, J. S., Unpublished work. 

11. Hill, F. S., Unpublished work. 



